Technical Field
This invention relates to computer system security, and more particularly, to a system and method for efficiently monitoring and securing a computer using an in-band monitor to intercept system calls, and using an out-of-band hypervisor to trap and respond to read and write requests of the CPU register(s) responsible for describing where control flow is transferred to when a system call is executed.
Background Information
Modern Operating Systems (OSs) manage the interaction between applications and resources of a system to facilitate efficient sharing as well as ensure overall stability and security. The OS component that performs these tasks is referred to as the kernel. It, among other things, schedules applications and processes resource requests. In order to do this, there is a mechanism, known as a system call, that signals the OS when an application desires access to a resource. A system call has a well-defined interface, and regardless of what an application is looking to do, it must conform to this definition in order to successfully communicate requests to the OS. Examples of these resources that leverage the system call interface are the file system, the network, the display, and the registry.
Because of its central role, security researchers, both offensive and defensive, understand that co-opting this interface has many benefits: For an attacker it can be used to provide stealth, hide files, processes, network sockets, and other resources so neither user nor security application can detect their presence. For a defender, manipulating this interface enables pervasive monitoring and can restrict access to resources. In the past, the common approach for attaching to this interface was through Direct Kernel Object Manipulation (DKOM). For example, on older Windows OSs (e.g. Windows XP 32-bit), function pointers in the System Service Descriptor Table (SSDT) would be replaced, thereby changing the behavior of the function when called. With concern over stability and compatibility, because multiple entities could hook and unhook the same entries, and anxiety over the legitimacy of performing these types of kernel changes, Microsoft introduced a technology known as PatchGuard™ (Microsoft Corporation. Redmond, Wash.).
PatchGuard operates in the context of the OS kernel and at certain points in time will verify the state of security critical objects (e.g. EPROCESS list and SSDT) and registers (e.g. Extended Features Enable Register Model Specific Register [MSR] and IDTR). (Model-specific registers (MSRs) are control registers provided by processor implementations to provide system software with features that are provided on specific processor implementations, but not others. Extended Feature Enable Register (EFER) is a register added to enable SYSCALL/SYSRET instruction.) When it notices a modification it triggers a fault, known as a “Blue Screen of Death (BSoD)” which will abort the current operation of the system and cause a system reboot. While this technology can be bypassed, it makes it much more difficult for attackers to install persistent rootkit style malware and has forced defensive security organizations to use other, “approved”, methods to implement their monitoring technologies. These “approved” methods, however, only provide a subset of the monitoring capabilities afforded through DKOM, such as modifying the SSDT. As an example, the “approved” methods allow for monitoring the registry, file system, network and process creation/destruction events but lack the ability to see other fundamental process interactions, such as calls to modify regions of a remote application's address space. The result is that modern monitoring capabilities are less robust than their pre-PatchGuard counterparts, and are insufficient for some forms of monitoring.
Thus, a need exists for improved monitoring capabilities that address the aforementioned drawbacks.